Skip to content

Refactor DuckDB utilites to improve SQL generation reliability#158

Merged
berggren merged 1 commit intomainfrom
duck-fix
Mar 4, 2026
Merged

Refactor DuckDB utilites to improve SQL generation reliability#158
berggren merged 1 commit intomainfrom
duck-fix

Conversation

@berggren
Copy link
Copy Markdown
Contributor

@berggren berggren commented Mar 4, 2026

Summary

Centralized DuckDB SQLite connection logic and implemented a validation-based retry mechanism for LLM-generated SQL queries.

Technical Details:

  • Connection Management: Introduced a _duckdb_sqlite_connection context manager in src/lib/duckdb_utils.py. This centralizes the loading of the SQLite scanner extension, security configurations, and database attachment, ensuring connections are consistently closed.
  • Validated SQL Generation: Modified generate_sql_query to include a retry loop (up to 3 attempts). The function now validates generated queries by executing them against the target database. If a query fails, the error message is passed back to the LLM to facilitate automated correction.
  • LLM Instruction Updates: Updated system prompts to enforce strict output rules, including ISO 8601 timestamp formatting using strftime, case-insensitive text matching with LOWER(), and specific handling for different epoch precisions.
  • Authorization Decorator Refactor: Restructured the require_access decorator in authz.py to provide separate sync and async wrappers. This ensures the access control logic executes correctly regardless of whether the decorated function is a coroutine.
  • Dependencies: Added pytz to pyproject.toml and poetry.lock to support timezone-aware date conversions within the generated SQL queries.
  • API Interface Change: Updated the generate_query endpoint in src/api/v1/files.py to pass the file object into the generation logic for query validation.

@berggren berggren requested a review from hacktobeer March 4, 2026 13:13
@berggren berggren merged commit 95c39b1 into main Mar 4, 2026
3 checks passed
@berggren berggren deleted the duck-fix branch March 4, 2026 13:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants